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human  observer.  Thus,  Lhe-hunin.'.*  visual  system  does  not  act  .like 
a spectral  analyser,  but  rather  appears  to  process  spatial  frequency 
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we  are  building  a new  graphics  display.  The  apparatus  will  permit  on-line 
control  of  the  amplitude  (contrast)  of  the  (X,Y)  frequency  (Fourier) 
components  that  make  up  the  texture  displayed.  Thus,  the  observer  will  be 
able  to  generate  a texture  that  appears  identical  to  another  texture 
having  a different  and  more  complex  spatial  frequency  content.  Of 
interest  is  the  minimum  number  of  spatial  frequency  components  necessary 
to  simulate  all  two-dimensional  textures.  If  we  find,  as  we  did  for 
one-dimensional  textures,  that  only  four  spatial  frequency  components 
arc  necessary,  then  v»c  may  proceed  to  design  a scheme  for  transmitting 
visual  information  ab- ut  textures  that  offers  a considerable  saving 
in  channel  capacity 
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Summary 

/I  Visual  textures  may  be  described  completely  by  their  spatial 
frequency  components.  For  one- dimensional  textures  whose  luminance 
varies  only  along  the  X-axis  of  the  display,  the  descriptive 
elements  are  gratings  that  have  sinusoidal  modulations  of  luminance. 
Although  any  arbitrary  oilmens ional  '‘blurred'*  texture  may  require 
a very'  large  number  of  sinusoidal  components  for  its  complete  physical 
description,  only  (components  are  needed  to  create  a texture 

that  appears  the  same  to  the  human  observer.  ThW  the  human  visual 
system  .does  not  act  like  a spectral  analyser,  but  rather  appears 
to  process  spatial  frequency  information  by  filtering  operations,  0* 
similar  to  that  performed  in  color  vision,  at  least  for  one-dimensional 
texture  patterns.  In  the  more  general  case,  textures  will  have 
luminance  distributions  varying  in  W-di»ensions  (i.e.>  both  X and  Y>. 
yn.yV.Ki  the  minimum  number  of  spatial  frequencies  required  to 

f simulate  dimensional  texture  patterns., we  are  building  a new 

graphics  display.  The  apparatus  will  permit  on-line  control  of  the 
amplitude  (contrast)  of  the  (X,Y)  frequency  (Fourier)  components 
that  make  up  the  texture  displayed.  Thus,  the  observer <wiil  be 
able-tt?  generate  a texture  that  appears  identical  to  another  texture 
having  a different  and  more  complex  spatial  frequency  content.  Of 
interest  is  the  minimum  nunfcer  of  spatial  frequency  components- 
necessary  to  simulate  all  two-dimensional  textures . ^If  we- find, 

— as  we  did  for  one-dimensional  textures.,  that  only  four  spatial 
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A computer- generated  texture  gradient 
size  increases  by  1.3  percent  from  on 
next  lower  row.  (Courtesy  of  A.  Polit 
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Experiments  In  Texture  Perception 


X.  Introduction 

Texture,  like  color,  is  one  of  the  primary  properties  of  an 
object  (Metzger,  1926;  Koffka,  1935).  Vet  our  knowledge  of  the 
texture  recognition  process  of  the  human  observer  is  meagre. 

Previous  studies  of  texture  may  be  crudely  divided  into  three 
categories; 

1. )  texture  gradients  as  shown  in  Fig.  1 and  their  roles 

In  slant  and  depth  perception  (Gibson,  1950;  Gruber 
and  Clark,  1956;  Uohlvill,  1962;  Flock  and  Moscatelli, 
1964;  Kraft  and  Winnlck,  1967); 

2. )  texture  discrimination  and  its  relation  to  the 

statistical  properties  of  the  display  as  illustrated 
in  Fig.  5 (Jones  and  Higgins,  1947;  McBride  and  Reed, 
1952;  Creen  et  al,  1959;  Stultz  and  Zweig,  1959; 

Julesz,  1962,  1965;  Pickett,  1962,  1964,  1967)  and 

3.)  the  search  for  continua  suitable  for  an  objective 
definition  of  "texture"  (Jones  and  Higgins,  1945; 
Rosenfeld,  1967;  Pickett,  1968;  Minsky  and  Papert, 

1969;  Julesz,  1971)  . 

Although  clearly  relevant  to  these  previous  studies,  our  primary 
approach  to  texture  perception  is  entirely  new  and  falls  into  still 
another  category.  The  novelty  of  the  new  approach  is  that  it  is 
concerned  only  with  describing  textures  that  appear  equivalent  to 
the  human  observer,  rather  than  trying  to  specify  the  physical 
characteristic  that  will  differentiate  between  all  textures.  The 


Figure  2 


Four  examples  of  one— dimensional  textures  composed  of  only 
a few  sinusoidal  components.  As  the  number  of  components 
increases,  the  textures  approach  the  middle  texture  of 
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atrempt  to  describe  equivalent  textures  therefore  is  quite  analogous 
to  development  of  color  science  where  the  primary  concern  was 

to  identify  spectral  compositions  that  would  appear  equivalent 
to  the  human  observer.  Such  energy  distributions  that  were  physically 
different  but  appeared  equivalent  were  called  ietamers.  Our  approach 
to  texture  perception  is  to  describe  such  metamers. 

Tlie  first  step  in  describing  color  equivalences  was  the 
recognition  of  the  dimension  of  wavelength  along  which  all  spectral 
compositions  co  ild  be  physically  represented.  Texture  may  also  be 
described  in  exactly  the  same  way  except  the  relevant  dimension  is 
now  spatial  frequency.  Thus,  if  at  the  onset  only  one-dimensional 
textures  are  considered  such  as  those  shown  in  Fig.  2,  then  Fourier’s 
theorem  states  that  any  such  texture  may  be  adequately  described 
by  the  magnitude  of  its  sinusoidal  components.  These  components 
are  of  course  merely  sine-wave  gratings  which  when  added  together 
in  suitable  proportions  will  physically  recreate  the  one-dimensional 
texture  pattern.  Thus,  the  dimension  of  spatial  frequency  is  used 
to  describe  all  possible  one-dimensional  textures  in  exactly  the 
same  manner  that  chromatic  wavelength  is  used  to  describe  all 
possible  colors. 

In  color  it  was  discovered  during  the  last  century  that  only 
three  suitably  chosen  wavelengths  were  needed  to  generate  equivalences 
to  all  possible  physically  realizable  colors  (Maxwell,  1855; 

Wright,  1928;  Guild,  1931).  This  property  of  color  equivalences 
is  imposed  by  the  fact  that  human  color  perception  is  based  upon 
the  energy  passed  through  only  three  independent  filters  each  having 
a different  wavelength  characteristic  (Stiles  and  Burch,  1959; 


Brown  and  Wald,  1964;  Marks,  Dobelle  and  MacNichol,  1964).  Until 
shown  to  be  otherwise,  it  is  also  possible  that  texture  perception 
could  follow  a similar  principle:  namely  that  all  one-dimensional 

textures  might  be  suitably  matched  by  only  a small  number  of 
suitably  chosen  sine-wave  gratings  put  together  in  the  right 
proportions.  Tills  research  is  a test  of  this  notion. 





II.  Ono-Dl  mens  tonal  Textures 

Textures  similar  to  those  In  Figs.  2 and  3 were  generated 
on  a PDP  11/20  computer  In  conjunction  with  a Kratos  display  scope 
(the  actual  display  was  not  grainy  as  in  the  present  illustrations). 

From  previous  experiments  (Richards  and  Polit,  1974),  we  had 
determined  that  four  suitably  chosen  spatial  frequencies  might 
match  all  one-dimensional  rextures.  Following  this  earlier 
paradigm,  we  chose  0.9,  2.3,  5.1  and  8.7  c/deg  as  our  primaries  for 
matching  all  sinusoidal  patterns.  Rather  than  attempting  to  match 
all  possible  patterns  in  the  texture  space,  an  additivity  assumption 
was  made:  that  any  pattern  may  be  described  by  the  linear 

superposition  of  its  Fourier  components.  Although  this  assumption 
regarding  the  behavior  of  the  visual  system  is  known  to  be  false, 
particularly  at  high  contrasts  (Davidson,  1968;  Cornsweet,  1970), 
the  approximation  is  good  at  low  contrasts.  With  this  approximation 
it  is  then  necessary  only  to  sped  y an  equivalence  between  each 
pure  sine-wave  pattern  and  the  chosen  primaries  in  order  to  specify 
matches  to  all  possible  textures.  The  procedure  is  thus  directly 
analogous  to  that  used  to  specify  color  matches  in  colorimetry 
(Wyszecki  and  Stiles,  1967).  And,  like  colorimetry  one  of  the 
primaries  will  always  be  added  as  a "desaturant"  to  the  test  frequency, 
with  the  combination  to  be  matched  by  the  remaining  two  primaries. 

When  a primary  is  added  as  a desaturant  to  the  test  frequency, 
the  primary  will  assume  negative  values. 

The  sine-wave  frequency  spectrum  was  then  sampled  from  0.2 
to  27  c/deg  with  the  contrast  of  the  test  frequency  held  fixed  at 
0.5.  Variable  amounts  of  contrast  of  one  cf  the  four  primaries 
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(0.9,  2.3,  5.1  or  8.7  c/deg)  were  then  added  to  the  chosen  test 
frequency.  The  test  frequency  together  with  its  (1  or  2)  primary 
desaturant  made  up  one  pattern.  The  second  pattern  consisted  of 
combinations  of  the  remaining  primaries  which  also  were  mixed 
together  in  variable  amounts  of  contrasts.  These  two  patterns 
appeared  aide  by  side  on  a Kratos  display,  and  the  observer 
could  control  the  amount  of  contraat  of  each  primary.  The  task 
of  the  observer  was  therefore  to  adjust  the  contraat  of  the 
primaries  (and  desaturant)  to  make  a texture  match  between  the 
two  fields. 

In  all  cases  a satisfactory  texture  match  could  be  found. 
Thus,  although  the  spatial  frequency  spectrum  for  both  patterns 
was  unequal,  the  textures  appeared  to  have  the  same  quality  to  the 
observer.  The  rsaulta  are  similar  to  those  reported  earlier  using 
a more  primitive  method  (Richarda  and  Polit,  1974).  Thus  we  may 
conclude  that  only  four  apatial  frequencies  are  required  to  match 
any  one  dlaensional  texture. 
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III.  TW>-Dlmenslonal  Textures 

Although  a considerable  amount  of  information  about  texture 
perception  may  be  obtained  using  one- dimensional  patterns,  any 
complete  description  of  texture  analysis  by  the  human  observer  must 
include  two  dimensional  textures  such  as  those  illustrated  in 
Figure  4.  In  order  to  generate  such  patterns  and  have  on-line 
control  of  their  sp.xtial  frequency  components,  we  have  designed 
and  are  building  a special  graphics  display.  This  display  will 
allow  us  to  generate  400  x 400  point  patterns  consisting  of  complex 
(computer-generated)  sinusoidal  modulations  of  luminance  that  may 
be  altered  every  20  asec. 

More  specifically .the  special  visual  display  under  construction 
consirts  of  9 subsystems  as  follows: 

1.  Monitors:  Conrac  SNA  17/C  (2) 

Monochrome  television  monitors 

2.  Operator  Controls:  TVo  channels,  each  with  independent 

control  of  three  sinusoidal  or  other  component 
amplitudes  and  the  a(x)*a(y)  product  term. 

Control  boxes  are  on  extension  cables  for 
convenience  and  flexibility  of  location. 

A six-channel  A/D  converter  digitizes  the 
control  settings  for  input  to  the  computer. 

3.  Function  Table  Computer:  A dedicated  PDP  11/10  Minicomputer 

is  used  to  monitor  the  operator  controls  and 
calculate  a(u)  and  b(u)  function  tables  in 
accordance  with  the  operator  control  settings. 


f 

I 


i 

i 


i 


- 11- 


Examples  of  two-dimensional  texture  patterns. 


I 


a(u)  ■ T.  A,sin(2irf.u  + 4 ) 
i - 1 1 1 


b(u)  ■ Z B sin(2nf,u  + 4.) 

i - 1 


4.  Video  Function  Generators:  Two  identical  custom 

designed  video  generators  are  provided  to 
store  the  computed  function  tables  and  generate 
a video  luminance  signal  of  the  form; 

La(X,Y)  - 1 + a(x)  + a(y)  + K^afx)  a(y) 

l^CX.Y)  - 1 + b (x)  + b(y)  + Kgbfx)  b(y) 

Provision  is  made  for  adding  an  external 
video  signal. 


1 


5.  Scan  Generator:  A custom  designed  digital  scan 

generator  will  generate  raster  coordinates, 
synchronizing  signals  and  control  signals. 

6.  Video  Refresh  System;  A custom-designed  video  refresh 

system  is  provided  to  allow  an  arbitrary 
two-dimensional  pattern  to  be  added  to  the 
texture  display.  The  refresh  system  employs 
a standard  core  memory  of  32,768  thirty-six 
bit  words  and  can  store  196,608  picture 
elements  (pixels)  with  6 bit  (64  level) 
gray  scale. 
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6.  Video  Refresh  System  (contd.): 

The  EMM  llicromemory  3000  series  has  been 
used  for  the  core  memory.  Four  3000DD 
(16K  x 18  bit)  cards  are  mounted  in  a 
5 1/A"  high  chassis  together  with  a control 
and  video  output  card,  power  supply  and 
cooling  fans. 

The  control  card  curcuit  provides  an  alternate 
m>de  of  operation  in  which  four  108  x 108 
checkerboard  patterns  can  be  stored  and 
refreshed . The  PDP-11/10  will  have  control 
of  mode  selection  and  can  select  which  of 
the  four  patterns  is  to  be  displayed. 

If  in  the  future,  the  video  refresh  capability 
should  be  no  longer  needed,  the  core  memory 
can  be  easily  converted  on  site  to  a general 
purpose  RAM.  EMM  offers  a Unibus  interface 
for  the  Micro  memory  3000  series. 

7,  Video  Interconnect  Panel:  A video  interconnect  panel 

is  available  to  permit  easy  and  flexible 
interconnection  of  video  signals.  Tht  panel 
also  contains  eight  adjustable  DC  voltage 
sources  and  a video  integrator  for  use 
with  the  special  effects  generator  and  video 


multiplexer. 
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8,  Special  Etfects  Ccnerator(s) : The  special  effects 

generator  produces  switching  signals  for 
split  screen  displays.  The  generator 
compares  x and  y scan  coordinates  with 
computer  controlled  set  points  and  will 
generate  a switching  signal  whenever  the 
scan  coordinates  are  within  a designated 
rectangular  space.  The  designated  space 
can  be  set  to  any  desired  height,  width  and 
location  with  a resolution  of  one  pixel. 

Thus,  the  generator  has  complete  flexibility 
and  can  even  be  used  to  generate  a "patch". 

For  future  expansion,  space  will  be  provided 
for  up  to  six  special  effects  generators, 
each  of  which  can  generate  one  "patch".  In 
addition,  a special  effects  interconnect 
panel  will  be  provided  to  allow  the  switching 
signals  to  be  combined  in  various  ways. 

9.  Video  Multiplexer:  An  eight  channel  video  multiplexer 

allows  switching  between  any  of  8 video 
sources.  The  multiplexer  switching  is  controlled 
by  one  or  more  special  effects  generators. 

This  capability  is  adequate  to  allow  up  to 
three  "patches"  to  be  overlaid  with  complete 
and  arbitrary  control  over  the  luminance  of 
all  eight  possible  combinations  of  the  three 
binary  variables. 


IV.  Random- Dot  Textures  (ulth  S.  Purksj 

Some  time  ago,  Juiesz  (1962)  proposed  that  two  textures 
created  by  random-dot  technic  s could  not  be  discriminated  if  they 
differed  only  in  their  3rd  or  higher  nth  order  statistic.  We 
have  continued  these  studies  and  have  found  several  examples  where 
4th  and  even  higher  nth  order  strings  are  discriminable , thus 
refuting  Julesz's  original  conjecture.  An  example  of  one  such 
texture  is  given  in  Fig.  5. 

We  are  at.  mpting  to  analyse  this  type  of  texture  pattern 
to  determine  the  basis  for  discrimination.  In  particular,  can  we 
relate  the  difference  in  any  way  to  a spatial  frequency  analysis 
performed  by  the  visual  system.  Clearly,  the  length  of  runs  of 
black  or  white  areas  is  also  important,  but  we  have  not  yet  resolved 
the  relation  between  run  length  and  spatial  frequency.  Our  final 
analysis  should  also  provide  Borne  insight  to  the  basis  for 
recognizing  texture  gradients  of  the  kind  depicted  by  Figure  1. 


Tw>  different  strings  differing  only  in  their  4th 
order  are  juxtaposed.  The  difference  between  the  left  and 
right  halves  is  visible,  contrary  to  earlier  proposals 
about  strings  differing  only  in  higher  order 
statistics. 
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V.  Texture  (Flow)  Gradients  (with  J.  Marroquln) 

A few  years  ago,  Glass  and  Perez  (1973)  described  some 
surprising  perceptions  of  flow  created  by  random  dot  interference 
patterns.  We  have  been  following  up  this  initial  report,  creating 
textures  from  random  arrays  that  appear  to  flow  . The  method 
is  quite  simple:  a random  dot  array  is  generated  by  the  computer, 
the  X,  Y position  of  each  dot  is  stored,  and  then  the  new  array 
is  transformed  and  superimposed  upon  the  old.  The  transformations 
we  have  been  working  with  primarily  are  expansions,  rotations  and 
spirals,  both  in  two  and  three-dimensional  space.  Our  intent  is 
to  determine  which  kind  of  transformations  are  visible  to  the 
observer,  i.e.,  appear  as  a pattern  of  flow  or  as  a texture 
gradient.  Clearly  these  studies  are  thus  exploring  the  more  complex, 
high-level  processing  of  the  human  observer.  Together  with  the 
previously  described  experiments  on  texture  discrimination  and 
texture  equivalences  (or  matching),  our  research  effort  is  thus 
covering  a broad  front  of  texture  analysis. 
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VI Pro  lections 

Texture  Is  one  of  the  primary  properties  of  an  object, 
l.ike  color,  texture  is  a quality  which  helps  the  human  observer 
to  define  and  identify  objects.  Yet  we  know  little  about  texture 
perception.  The  research  in  progress  offers  a completely  new 
approach.  The  most  important  a ct  of  the  research  is  that  the 
texture  analysing  mechanism  of  the  human  observer  is  only  four 
dimensional.  Thus,  all  (one-dimensional)  textures  may  be  completely 
specified  in  terms  of  only  fou-  primaries.  Such  a specification 
will  describe  all  equivalences  between  textures.  This  is  a non- 
trivial accomplishment.  In  the  domain  of  color  perception,  if  it 
were  necessary  to  describe  all  colors  in  terms  of  its  precise 
wavelength  composition,  then  the  transmission  of  chromatic  information 
would  not  have  became  a feasible  possibility.  The  fact  that  the 
human  observer  filters  the  wavelength  spectrum  allows  us  to  build 
economical  conmunication  systems  for  chromatic  information.  By 
the  same  token,  if  it  may  be  demonstrated  that  the  human  observer 
analyses  textures  on  the  basis  of  only  a few  filters,  then  a 
considerable  saving  in  the  transmission  of  texture  information 
may  be  gained.  This  practical  benefit  far  outweighs,  but  in  no 
way  diminishes  the  further  gains  that  we  will  achieve  in  our 
understanding  of  the  human  visual  system. 
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S pedal  Visual  Display  'ystem  Components  and 
Connections 

Video  Refresh  Subsystem  Block  Diagram 
Scan  Generator,  Block  Diagram 
Video  Generator,  Block  Diagram 
Special  Effects  Generator 

Sped  xl  Visual  Display  System,  Block  Diagram 
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